Image processing apparatus, control method of image processing apparatus, program and storage medium

ABSTRACT

During a display of a slideshow of a composite image comprising a plurality of images, a user operates a print bookmark, resulting in composite images and individual images that can be printed being set and stored as print candidate images. By displaying, in a list; stored candidate images on a display unit as print candidate images, an image processing apparatus is provided which enables the user to specify a desired image for printing.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus handlingcomposite images each comprising a plurality of images, control methodof an image processing apparatus, program, and storage medium.

2. Description of the Related Art

Recently, as a result of the popularization of digital cameras,opportunities to view image data shot by a digital camera at home on apersonal computer, television receiving machine, etc., are increasing.In this case, as a viewing method, there is proposed displaying a singleimage data selected by a user, or displaying a slideshow display of aplurality of image data which are successively shown and switched at apredetermined time.

Further, recently, there is proposed displaying a slideshow displayhaving complicated display functions such as simultaneously displaying,switching, superimposing, etc., a plurality of image data as a method ofdisplaying image data in a slideshow display. In Japanese PatentLaid-Open No. 2003-196283, there is disclosed an invention relating toan image forming program in which it is possible to display an imagedata and partial image data resulting from extraction of one part of theimage data, and perform printing. However, in this invention disclosedin Japanese Patent Laid-Open No. 2003-196283, an image data region thatdoes not change and is fixed is specified, and the region is displayedon a display apparatus or recorded in a printing medium.

In addition, in the case that a plurality of image data issimultaneously displayed, there is already proposed specifying imagedata in a still state and displaying or printing the image data.

In a slideshow display having complicated display functions such assimultaneously displaying, switching, superimposing, etc., a pluralityof image data as a method of displaying image data in a slideshowdisplay, it is desirable that it is possible to specify and displayspecific image data, and print the image data. That is, specifyingimages to be printed is made easier even when a user is in the middle ofexecuting a slideshow display, if it is possible to display or print aparticular image data by specifying the image data from among aplurality of image data included in the slideshow display. In this case,convenience to a user is increased if it is possible to specify an imagedata even when the image data is changing dynamically.

SUMMARY OF THE INVENTION

Therefore, the object of the present invention is to provide a systemwhich makes it possible to specify a particular image data from acomposite image data comprising a plurality of image data, and to enabledisplaying and printing of the specified particular image data.

According to an aspect of the present invention, there is provided animage processing apparatus, comprising: a display control unitconfigured to display on the display apparatus a composite imagecomprising a plurality of images while dynamically changing theplurality of images; an receiving unit configured to receive outputcommands during displaying of the composite image on the displayapparatus by the display control unit; and a selection unit configuredto display, so that a user can select as an image to be output, eitherthe composite image at a timing before or after a timing at which theoutput command has been received by the receiving unit.

According to another aspect of the present invention, there is provideda control method of an image processing apparatus, comprising:controlling step of controlling a composite image comprising a pluralityof images by displaying on a display apparatus while dynamicallychanging the plurality of images; receiving step of receiving an outputcommand from a user while displaying the composite image on the displayapparatus by the controlling step; and selecting step of displaying thecomposite images at a timing before and after the timing at which theoutput command has been received by the receiving step so that a usercan select either one of the composite images as an image to be output.

In order to achieve the above object of the present invention, stillanother embodiment of the present invention provides acomputer-executable program describing a processing procedure of acontrol method of the image processing apparatus.

In order to achieve the above object of the present invention, yetanother embodiment of the present invention provides a computer-readablestorage medium in which a program describing a processing procedure of amethod of controlling the image processing apparatus is stored.

The present invention enables specifying a particular image data from acomposite image data comprising a plurality of image data, and to enabledisplaying and printing of the specified particular image data.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing the entire system configuration of acomposite image processing apparatus according to an embodiment of thepresent invention.

FIG. 2 is a block diagram showing a TV receiving apparatus used in acomposite image processing apparatus according to an embodiment of thepresent invention.

FIG. 3 is a diagram showing an operating side of a remote controlapparatus controlling a TV receiving apparatus used according to anembodiment of the present invention.

FIG. 4 is a diagram showing a strip-shaped type slideshow displayexecuted according to an embodiment of the present invention.

FIG. 5 is a diagram showing a hierarchical structure of software of asystem control unit used according to an embodiment of the presentinvention.

FIGS. 6A and 6B are diagrams showing a display data table used accordingto an embodiment of the present invention.

FIGS. 7A-7C are display state tables and diagrams showing display statesin a strip-shaped type slideshow display executed according to anembodiment of the present invention.

FIGS. 8A-8D are display state tables and diagrams showing display statesin a randomly superimposing type slideshow display executed according toan embodiment of the present invention.

FIGS. 9A-9D are diagrams showing registration of print candidate imagesin a strip-shaped type slideshow display executed by an embodimentaccording to the present invention.

FIG. 10 is a diagram showing a state of displaying in a list registeredprint candidate images after finishing displaying of a strip-shaped typeslideshow display executed according to an embodiment of the presentinvention.

FIGS. 11A-11D are diagrams showing registration of print candidateimages in a randomly superimposing type slideshow display executedaccording to an embodiment of the present invention.

FIG. 12 is a diagram showing a state of displaying in a list registeredprint candidate images after finishing displaying of a randomlysuperimposing type slideshow display executed according to an embodimentof the present invention.

FIG. 13 is a flowchart diagram showing a registration operation of aprint candidate image according to a first embodiment of the presentinvention.

FIG. 14 is a flowchart diagram showing a registration operation of aprint candidate image according to a first embodiment of the presentinvention.

FIG. 15 is a flowchart diagram showing a printing operation according tothe first embodiment and a second embodiment of the present invention.

FIG. 16 is a flowchart diagram showing a registration operation of aprint candidate image according to the second embodiment of the presentinvention.

FIG. 17 is a flowchart diagram showing an operation according to otherembodiments of the present invention.

DESCRIPTION OF THE EMBODIMENTS

An image processing apparatus according to an embodiment of the presentinvention will be described below using a television broadcast receivingapparatus (hereinafter referred to as TV receiving apparatus) 100 as anexample.

FIG. 1 is a schematic view of a system in which a printing apparatus 200is connected to a TV receiving apparatus 100, which is an imageprocessing apparatus according to a first embodiment of the presentinvention. The TV receiving apparatus 100 has an antenna 180, receives abroadcast signal from a broadcast station, and obtains a video signal,audio signal, and other information signals. Then, the video signal,audio signal, and other information signals are output to a display unit110 (details described below) of an LCD panel, CRT, etc., as well as anaudio output unit 111 such as a speaker. Basically, the TV receivingapparatus 100 performs various controls by receiving signals from aremote control apparatus 150. Of course, it is possible, depending onthe situation, to receive a broadcast signal of a broadcast station froma cable of a cable TV company without using an antenna 180.

In addition, a display unit 110 displays not only broadcasted video, butalso video signals of other AV devices and image signals accumulated ina memory described later, as well as a user interface of a variety ofcontrols using a remote control apparatus 150. Therefore, the displayunit 110 is used as a user interface when carrying out printing controlby the printing apparatus 200 which has been connected. The TV receivingapparatus 100 and the printing apparatus 200 are linked by a USB cable140. Implementation of linking of a TV receiving apparatus 100 and aprinting apparatus 200 is possible under both wired and wirelessconditions. A USB connection is possible in the case of a wiredconnection, while a wireless LAN connection or IR communicationconnection is possible in the case of a wireless connection. In thediagrammed example, the TV receiving apparatus 100 and the printingapparatus 200 are connected with the USB cable 140. In addition, the TVreceiving apparatus 100 is connected to the Internet via a network cable145 and a router 146. Therefore, the TV receiving apparatus 100 isconfigured so that information on the Internet can also be viewed. Inaddition, the printing apparatus 200 can also be connected to the TVreceiving apparatus 100 via the network cable 145.

Next, a circuit block of the TV receiving apparatus 100 will bedescribed in detail with reference to the function block diagram in FIG.2. FIG. 2 is a functional block diagram, configured as the firstembodiment of the present invention, showing a circuit configuration ofthe TV receiving apparatus 100. In FIG. 2, reference numeral 101 is atuner unit which amplifies a high-frequency signal of a receivedtelevision broadcast (hereinafter referred to as TV broadcast), performsselection of a broadcast channel desired to be received, performsdemodulation, and outputs TS data in which video/audio program data,various data, etc., have been mixed.

In addition, reference numeral 102 is a data demultiplexing unit, whichdemultiplexes TS data that has been output from the tuner unit 101 intovideo/audio program data, program information data described below, datafor data broadcasting (data broadcast content, various metadata, controldocuments, and other data for data broadcasting), etc. Demultiplexedvideo/audio program data is supplied to an AV decoder unit 103 describedbelow, and other program information data and data for data broadcastingare output to a memory unit 105 through an internal bus 104.

The AV decoder unit 103 decodes video/audio program data supplied fromthe data demultiplexing unit 102, and video/audio data encoded in MPEG,etc., supplied from the memory unit 105 through the internal bus 104.

Audio data which has been output from the AV decoder unit 103 issupplied to an audio control unit 107, and, after being mixed with otheraudio data if required, the audio data is output from the audio outputunit 111 as audio. In addition, video data output from the AV decoderunit 103 is displayed on the display unit 110 after being composed withother video data, character data, graphic data, etc., which are to besuperimposed, in a display superimposing control section 108.

In the audio control unit 107, audio data from the AV decoder unit 103and audio data from the internal bus 104 are switched or composed andoutput to the audio output unit 111 after controlling volume, soundquality, realistic feeling, etc.

In the display superimposing control section 108, video program datafrom the AV decoder unit 103 and image data of a digital camera, etc.,generated in a graphic control unit 109, are processed based on controlby a system control unit 112. That is, the display superimposing controlsection 108 performs switching, composition, enlargement/reduction, andvarious other conversions and corrections on a plurality of image data,and outputs to the display unit 110.

The graphic control unit 109 generates, based on control by the systemcontrol unit 112 described below, GUI screens, etc., such as a databroadcast screen, an screen of an application, installed in inside theTV receiving apparatus 100, for viewing a digital camera, and a message.

The display unit 110 displays a video/image data output from the displaysuperimposing control section 108, and includes a display apparatushaving, for example, a large screen and a high picture quality by havinga resolution of 1920×1080 pixels, a frame rate of 60 Hz, and a size of50 inches.

The memory unit 105 performs temporary storage, etc., of data broadcastcontent, program information data, etc., demultiplexed by the datademultiplexing unit 102. Further, the memory unit 105 performs aslideshow display or temporary storage, etc., of expansion data of adigital camera image for printing, a display data table, a display statetable, a print candidate buffer, and other control data in the TVreceiving apparatus 100. In addition, the memory unit 105 can also beused as work memory by the system control unit 112 in order to executeprograms.

Reference numeral 113 is a data accumulation unit, which comprises, forexample, an HDD apparatus, and which is also used to store receivedvideo/audio program data, image data of a digital camera accumulated ina memory card 115, and so on. In addition, a print candidate listserving as a print candidate specified for printing during a slideshowdisplay (hereinafter referred to as printing bookmark) is also stored inthe data accumulation unit 113. Also, a plurality of display data, etc.,starting with an OSD (On Screen Display) and a GUI (Graphical UserInterface) are stored in the data accumulation unit 113.

Reference numeral 116 is an interface unit connecting the printingapparatus 200, and transferring data to the printing apparatus 200 whenperforming printing of various data.

Reference numeral 114 is an external memory connection unit comprising aconnector unit for connecting the memory card 115 of a digital camera,etc.

The memory card 115 comprises a non-volatile memory of a semiconductorused for the accumulation of image data of a digital camera, etc., andhas accumulated on it image data of a digital camera, etc., forperforming a slideshow display or printing in the TV receiving apparatus100.

Reference numeral 117 is an Internet connection unit, and, for example,connects to the Internet via the router 146, and performs transmissionand reception of various data by connecting to a broadcast stationserver, accounting server, etc. Of course, it is possible to view, etc.,other various information on the Internet.

Reference numeral 118 is a remote control apparatus control unit whichperforms remote control transmission and reception using infrared lightwith a remote control apparatus 150, and transfers the received data tothe system control unit 112.

The system control unit 112 has overall control over each unit insidethe TV receiving apparatus 100. The system control unit 112 comprises aCPU, main storage memory, bus control unit, program storage memory,parameter storage memory, ID storage memory, clock unit, timer unit,etc. In addition, the tuner unit 101 performs switching of a receivingchannel through control of the data demultiplexing unit 102, and exertscontrol so that signals from a desired broadcast station are received.

In addition, the system control unit 112 demultiplexes video/audioprogram data, program information, data for data broadcasting, etc.,and, while displaying a program, generates data broadcast and EPG(Electronic Program Guide) data. Further, the system control unit 112controls the graphic control unit 109, and controls so that display ofdata broadcast, EPG, etc., is performed. In addition, the system controlunit 112 reads out image data of a digital camera, etc., from the memorycard 115, and controls so that they are shown in a slideshow display.Details will be described below.

In addition, the system control unit 112 also performs display of otherapplications inside the TV receiving apparatus 100, and control ofdisplaying of a message, operation panel, etc. When there is a requestto print image data of a digital camera, etc., the system control unit112 recognizes and controls the printing apparatus 200 connected to theinterface unit 116, and prints the requested image data.

Further, the system control unit 112 controls the display superimposingcontrol section 108, performs composition of image data from the graphiccontrol unit 109, etc., and various information display processing,etc., of correction, etc., and displays on the display unit 110. In thesame way, the system control unit 112 controls the audio control unit107, performs composition of audio and control of output, and performscontrol of output to the audio output unit 111. In addition, the systemcontrol unit 112 controls transmission and reception of data with theremote control apparatus 150 operated by the user, who is a viewer, and,based on the data, performs control within the TV receiving apparatus100. The internal bus 104 is a data and control bus, and, as describedabove, is a bus set up in the TV receiving apparatus 100 for use intransferring image data and audio data, as well as in transferring ofinformation of each department.

FIG. 3 shows details of an operating side of a remote control apparatus150 used together with the TV receiving apparatus 100, which is anembodiment of the present invention. Key buttons of a remote controlapparatus 150 are a power key 151, a volume key 152, a channel key 153,etc., which are keys used at the time of normal TV viewing. In addition,the remote control apparatus 150 has a data broadcast key 154(abbreviated “d key” hereinafter) for use in transitioning to a databroadcast, as well as buttons for operation of applications of the TVreceiving apparatus 100, such as a menu key 155, a data broadcast, andan application. Further still, the remote control apparatus 150 has aGUI, a cursor and enter key for performing operations of an operationpanel, etc., as well as a cancel key 156. In addition, 157R through 157Bare red, yellow, blue, and green keys, which are color keys used asfunction keys in the TV receiving apparatus 100. In the TV receivingapparatus 100 according to the present embodiment, they are used in aslideshow display and in setting printing images.

The red key 157R is used as a key for performing print bookmarking whileslideshow display is in operation, but it is possible to switch tostarting a direct print menu by switching. In addition, the remotecontrol apparatus 150 has a display unit 160, and displays the time andvarious messages. The remote control apparatus 150 transfers a key codethrough the remote control apparatus control unit 118 of the TVreceiving apparatus 100.

FIG. 4 is a screen in the middle of a slideshow display, and, asdescribed below, is an image representing execution of a strip-shapedtype slideshow on the display unit 110. In FIG. 4, strip-shaped imagedata A, B, C, and D are displayed. In addition, on the bottom side ofthe strip-shaped image data A, B, C, and D is a user interface which canbe operated by the remote control apparatus 150. In doing so, functionsof the red key 157R, yellow key 157Y, blue key 157B, and green key 157Gare displayed during slideshow display.

FIG. 5 is a diagram showing the hierarchical structure of softwaredivided per each control unit of the system control unit 112 of the TVreceiving apparatus 100. Reference numeral 501 is a basic system controlunit including an OS, and under the management of the basic systemcontrol unit 501, other control units form a hierarchical layerstructure in which a slideshow display control unit 502, a printcandidate generation unit 503, a print control unit 504, and a differentcontrol unit 505 operates.

The slideshow display control unit 502 generates a display data tabledescribed below based on operations and settings by a viewer, and, basedon the display data table, generates a display state table describedbelow. The print candidate generation unit 503 generates a printcandidate list based on operation of a print bookmark, which is anoutput command from the viewer, during a slideshow display, and storesthe print candidate list in a data accumulation unit 113. The printcontrol unit 504 selects print image data from the print candidate listbased on viewer operation. Then, the printing apparatus 200 iscontrolled through the interface unit 116, and prints image data of adigital camera, etc., accumulated in the memory card 115 requested bythe viewer.

The different control unit 505 includes all control units including theTV receiving apparatus 100 capable of receiving a TV broadcast,controlling a data broadcast, controlling communication, controlling thedisplay, controlling the remote control, and so on.

In the slideshow display described above, a display control unit insidethe different control unit 505 reads out image data accumulated in thememory card 115 based on the above display state table. Then, thedisplay control unit controls the graphic control unit 109, the displaysuperimposing control section 108, and the slideshow display.

FIGS. 6A and 6B are display data tables in which are described imagedata to be displayed and display information, control information, andprint setting information during slideshow display. The display datatable of FIG. 6A includes a common display information 601 of aslideshow display, while FIG. 6B includes an individual displayinformation 602 of each image data. The common display information 601comprises the following five items: image display format, displayinterval, display order, transition effect and print mode. In theseitems are reflected the contents set by the user using the remotecontrol apparatus 150 before the start of displaying a slideshow.

Besides displaying a normal slideshow in which one image per one screenis successively displayed, other slideshow display formats are possible,such as displaying a strip-shaped type slideshow with two or four imagesper one screen. Displaying a strip-shaped type slideshow means randomlyextracting one image data, and lining up in a strip-shaped form anddisplaying the extracted image data as described in FIG. 4.

According to a first embodiment of the present invention, as in adisplay state 703 shown in FIG. 7C described below, the display screenof the display unit 110 is divided vertically into four, and images aredisplayed in order starting with image A. Described below is the case ofperforming operations to display a strip-shaped type slideshow so that,when all four images have been displayed, the next image overlaps thefirst image. A randomly superimposing type slideshow display is adisplay format in which, as in a display state 803 described below andshown in FIGS. 8C and 8D, a plurality of images are strewn across theentire screen and displayed. The display position of each image israndomly computed and decided by the slideshow display control unit 502.

In addition, these display intervals are time intervals until startingdisplay of the next image in a slideshow display. During these timeintervals, the display is in a pause state. The display interval is setaccording to user preferences. According to the embodiment of thepresent invention, the time interval is set to 3 seconds. The displayorder of images desired to be in the slideshow display can be selectedfrom among file name order, shooting date order, and random.

A transition effect is a manner in which an image appears whendisplaying the next image after the present image. According to theembodiment of the present invention, transition effects such asslide-in, dissolve, and wipe are available.

A print mode is a setting mode for automatically selecting printcandidate images when performing print bookmarking for printing in aslide show display. It is possible to select from among three printmodes: print candidate selection mode 1, print candidate selection mode2, and direct print mode.

Here, the print candidate selection mode 1 is based on the assumptionthat the image which the user wishes to print is among the imagesdisplayed while the print bookmark button is being pressed. And, duringprint bookmarking, if the slideshow display is not dynamically changing,each image comprising that composite image, as well as the entireslideshow, becomes a print candidate. In addition, if the slideshowdisplay is dynamically changing, each image comprising each slideshowdisplay before and after change, as well as the entire slideshow(composite image) becomes a print candidate. That is, a composite imageincluding images that have been newly added and displayed by a dynamicchange, and a composite image before the dynamic change and do notinclude this added image are included as print candidates.

The print candidate selection mode 2 is based on the assumption that, inaddition to the above print candidate selection mode 1, when printbookmarking, there is at least one of each image which comprises thedisplay in an entire slideshow display other than the current slideshowdisplay. That is, all other slideshow displays in which each “currentlyviewable” image at the time the print bookmark button (which is anoutput command) is pressed are print candidates. In this case, not onlyslideshow display images before the time the print bookmark button ispressed is considered to be a print candidate. That is, after the timeat which the print bookmark button has been pressed, and the slideshowdisplay has advanced, slideshow display images up until the time thatall images that were displayed until the time of print bookmarking are“hidden and cannot be seen, or disappeared” are also print candidates.

The direct print mode is a mode of performing printing, at the time abookmark key has been pressed, by pausing displaying of a slideshow,making the image immediately before and the image included in it a printcandidate, and starting a print menu screen.

On the other hand, as an individual information of each individual imageof a display data table, there is described for each image, as shown inreference numeral 602 of FIG. 6B, image file name, storage location,display order of each individual image, extraction location, extractionsize, overlap with other images when displaying, display location, imagerotation angle, and magnification ratio. These settings have beencomputed by the slideshow display control unit 502, before starting aslideshow display, based on settings of display state input by a userusing the remote control apparatus 150 as well as images selected to bedisplayed.

FIG. 7B is a display state table 702 in the case of a strip-shaped typeslideshow display. FIG. 7C is the display state 703 of a display screen.The display state table 702 describes images (all images other thanthose images which are below the other images and which are notdisplayed) related to the present display inside the individual displayinformation 602 as well as their state. As a result, to the displaystate table 702 of FIG. 7B is added the items of display change anddisplay state. One image of the slideshow display is generated accordingto the display state table 702. In addition, FIG. 7A is a common displayinformation 701 of the slideshow display.

In the display state table 702 of FIG. 7B, display change indicateswhether the display is in the middle of change. A value of “C” fordisplay change indicates that the “display is in the middle of change”.“Display is in the middle of change” is the case of “about to displayfrom where there are no images”, “viewable images are about to bedeleted”, or “viewable images are about to become invisible by beingoverlapped by display of other images”. In addition, if the value ofdisplay change is “F”, this indicates that the “display is notchanging”.

Further, the display state determines whether an individual image whichcomprises composite images of a slideshow display are displayed on ascreen, using the standards below. That is, if the display state is “V”,the individual image is in a “viewable” state. This “viewable” state ina display screen (when in a display state, a display screen at the timeof finishing the change operation) based on the display state table 702is as indicated below. That is, determination is made based on thestandard above a certain level (above 20% according to the presentembodiment) of the face portion of an image is shown, or above a certainlevel (above 50% according to the present embodiment) of the entireimage is displayed.

In addition, if the display state has a value of “H”, the image is in a“hidden and cannot be seen, or cannot be seen because disappeared”state, and indicates cases other than the above “viewable” case.

The display state table 702 of FIG. 7B, when starting displaying of aslideshow, is generated in the memory unit 105 inside the TV receivingapparatus 100 by the slideshow display control unit 502, and isrewritten every time there is a dynamic display change by the slideshowdisplay control unit 502. Therefore, the newest display state is alwaysbeing shown, and the display superimposing control section 108 controlsthe display of the display unit 110 along the display state table.

FIG. 7B shows changes of the display state table 702 starting from thetime point where there is no image is displayed, then image A isdisplayed at the D1 time point and finally image F is displayed at theD10 time point. FIG. 7C shows the display state 703 of the screen in sodoing. Images at D1, D3, D5, D7 and D9 which have a diagonal line areimages (display change “C”) that are going through display changes. Asindicated in the common display information 701 of FIG. 7A, the displayinterval is set to 3 seconds, the display order is set to file nameorder, the transition effect is set to slide-in, and the print mode isset to print candidate selection mode 1. Therefore, there is a 3-secondperiod, during which dynamic display is in a pause state, until thestart of the next display of the new image B, image C, and image F fromthe D2, D4, and D8 time points.

FIGS. 8A-8D show a common display information 801, a display state table802 of a randomly superimposing type slideshow display, and a displaystate 803 of a display screen.

The present embodiment shows a composite image starting from the timepoint at which image A and image B of D′1 has been displayed to the timepoint at which image I of D′11 has been displayed. In this randomlysuperimposing type slideshow display, the display state table 802 ofFIGS. 8B and 8C is generated as described above. As in FIG. 7C, imageswith diagonal lines are images that are in the middle of change (displaychange “C”). In FIG. 8D, after the D′7 time point, it is difficult toshow in the drawings the image state of “display is in the middle ofchanging”, so this has been omitted.

FIG. 9A through 9D show images registered to the print candidate list asprint candidate images, in the case where the user has pressed the printbookmark button, in the display state at the time points D4, D5, D6 andD9 during the operation of the strip-shaped type slideshow display ofFIG. 7C. That is, in the case where the print bookmark button has beenpressed at the D4 time point, image A, image B, and the composite imageof image A and image B become the print candidate image. The compositeimage of the actual image A and image B are displayed in the left halfof the display unit 110, but during printing, a composite image whichhas been stretched in the width direction becomes the print candidateimage. Therefore, at time points D4, D5 , D6 , and D9, 3, 5, 4, and 7print candidates screens, respectively, are registered.

FIG. 10 shows an example of an image which is displayed on the displayunit 110 of the TV receiving apparatus 100 when performing printingoperation, after finishing slideshow display by pressing the printbookmark button at the time of displaying at the two time points D4 andD9 of FIG. 7C. Here, the print candidate images registered at the D4time point and the D9 time point are displayed in a list so that it ispossible for a user to select an image to be printed as an option.

FIGS. 11A through 11D show display images registered to the printcandidate list as print candidate images in the case where the printbook mark button has been pressed in the display state of the timepoints D′1, D′5, D′6 and D′8, during the operation of a randomlysuperimposing type slideshow display of FIG. 8D.

FIG. 12 shows a list of images which are print candidate images whichwere displayed on the display unit 110 and which were registered whenthe print bookmark button was pressed twice when displaying the D′1 andD′8 of FIG. 8D.

Next, a buffer memory related to a table used in the present embodimentwill be described. A print candidate buffer is for temporarily storing adisplay state table which has the possibility of being a printcandidate, and is generated in the memory unit 105 by the slideshowdisplay control unit 502. In addition, a display image list is a listfor registering individual images (those with display state “V”),namely, print candidate images, determined to be the “viewable” imagesinside slideshow display images. These are generated in the dataaccumulation unit 113 as required when a print bookmark button has beenpressed, and are used together with a print candidate list at the timeof displaying print candidate images. Further, a print candidate list isfor selecting and storing, from a print candidate buffer, a displaystate table including images which are print candidates, by the printcandidate generation unit 503. The print candidate list is generated inthe data accumulation unit 113 and is used when printing.

First Embodiment

Next, an operation according to a first embodiment, in the case of usinga strip-shaped type slideshow display, will be described using FIGS.7A-7C and FIG. 13 through FIG. 15.

After a user has set, using the remote control apparatus 150, the imagedata and the parameters such as the display format of the slideshowdisplay, transition mode, and display interval, slideshow display isexecuted. FIG. 13 shows the operation flow of control of a slideshowdisplay.

After starting processing, the slideshow display control unit 502confirms a setting value at Step S1301, and generates a display datatable at Step S1302. At Step S1302, a display state table for displaycontrol is generated based on the generated display data table, and atStep S1303 a determination is made as to whether it is a rewrittentiming based on a display interval in the common display information701. If it is a timing at which rewriting occurs, the process advancesto Step S1304 and rewriting occurs. If not at a timing at whichrewriting occurs, waiting occurs until it is at a timing at whichrewriting occurs. Therefore, during this interval, dynamic display is ina pause state.

Next, at Step S1305, the rewritten display state table checks the stateof “display change” and, if “there is no display change”, that is, ifthe display state table is comprised by “display change ‘F’ state”, theprocess advances to Step S1306. Then, the display state table is alsostored in the print candidate buffer. However, if there is a displaychange, that is, if there is even one “display change ‘C’”, storage tothe print candidate buffer is not performed. After storing the displaystate table in the print candidate buffer, the process advances to StepS1307, and a determination is made as to whether a slideshow displayends. If not, the process returns to Step S1302, the next timing atwhich the display is rewritten is determined, and rewriting to the “nextdisplay state table” is performed at Step S1304. Then, at Step S1306,the “next display state table” is additionally stored in the printcandidate buffer. This processing sequence is performed until slideshowdisplay is finished.

In displaying a slideshow, a display control unit within another controlunit 505 detects that a display state table has been rewritten, and,based on the display state table, performs display control and performsslideshow display.

FIG. 14 is an operation flow of the print candidate generation unit 503,and this processing routine starts processing when slideshow display isstarted, and pressing of the print bookmark button (e.g., the red key157R of the remote control apparatus 150) at Step S1401 is first waitedfor.

Here, when a user who is viewing a slideshow presses the print bookmarkbutton (for example, red key 157R of the remote control apparatus 150)to perform print image registration, this is received as an outputcommand. Then, the print candidate generation unit 503 confirms thesetting value of the slideshow display at Step S1402. And, if at StepS1403 the print mode is determined to be a print candidate selectionmode 1, the newest display state table registered in the print buffer atStep S1404 is registered in the print candidate list. Next, the processadvances to Step S1405, and whether the current display state is in themiddle of “display change” is checked. If in the middle of a “displaychange”, the process advances to Step S1406, at which there is waitingfor generation of storage operations to the print buffer from thedisplay state table. Here, if a storage operation is generated, theprocess advances to Step S1407, and the stored display state table isregistered in the print candidate list. After registration, the processadvances to Step S1408, and whether displaying of the slideshow isfinished is confirmed. If not finished, the process returns to StepS1401, and the next pressing of the print bookmark button is awaited.

At Step S1405, in the case of not being in the middle of a “displaychange”, the process advances to Step S1408, and it is confirmed whetherslideshow display is finished. If it is not finished, the processreturns to Step S1401, and the next pressing of the print bookmarkbutton is awaited. In addition, by the determination at Step S1403, ifthe case is not print candidate selection mode 1, the process advancesto Step S1409, and processing of a different mode is performed.

In addition, in the case where the print bookmark button has beenpressed at a timing other than Step S1401 of this operation routine, theprocess at any time returns to Step S1401, and restarts operations. Inthis way, viewing of the slideshow and selection of images desired to beprinted is advanced.

FIG. 15 is an operation flow of the operation routine when performingprinting. Slideshow display is finished after finishing selection ofimages desired to be printed, and the user, in order to print imagesthat have been print bookmarked using the slideshow display, uses theremote control apparatus 150 to open the menu screen at Step S1501 anddisplays the print menu. Further, when a button to display a list ofprint candidates is pressed, the process advances to Step S1502, anddisplay of the list of print candidates is performed. Then, further, theprocess advances to Step S1503, at which there is a check of print statetable by the print candidate list, and the print candidate image isdetermined.

Here, FIGS. 9A-9D will be referenced in describing the determination ofthe print candidate when the print bookmark button is pressed at aplurality of timings.

FIGS. 9A-9D show print candidate images at, respectively, the timepoints D4, D5, D6 and D9 of FIG. 7C. in the case that the print bookmarkbutton has been pressed. The images containing the

mark on the right side of D4, D5 and D9 are slideshow display images ata timing at which the print bookmark button has been pressed. An X mark,added for description purposes, are composite images that are not printcandidate images.

As described before, FIG. 9A shows the case in which the print bookmarkbutton has been pressed at the D4 time point of FIG. D4. In this case,the content (the display state table 702 at the D4 time point of FIG.7C) of the newest print candidate buffer at the D4 time point is storedin the print candidate list. This D4 time point is a timing at whichthere is no display change. Therefore, the individual images (image Aand image B) inside the display state table 702 at the D4 time point,and the composite image of image A and image B of the slideshow display,comprising the display state table 702, become print candidates withoutwaiting for the print candidate buffer to be stored again.

In this case, if the composite image of the slideshow display is to bemade the print candidates without change, it becomes the image on theright with the X mark. However, in the case of displaying a strip-shapedtype slideshow, the print candidate image will correspond to a compositeimage that has been stretched sideways, as without the stretching, thetotal balance will be poor.

FIG. 9B shows the case of pressing the print bookmark button at the D5time point of FIG. 7C. In this case, the newest print candidate buffercontent (the display state table 702 of the D4 time point of FIG. 7C) atthe D4 time point will first be stored in the print candidate imagelist. This D5 time point is a timing of display changing, so storage tothe print candidate buffer of the display state table 702 at the next D6time point is awaited, and the stored newest print candidate buffercontent (the display state table of the D6 time point of FIG. 7C) isalso registered in the print candidate image list.

FIG. 9C shows a print candidate image corresponding to the D6 timepoint, and the print candidate image of the D5 time point of FIG. 7C isan image included in either the print candidate image at the D4 timepoint, or at the D6 time point. That is, it becomes like FIG. 9B, andthe image (the image with the X mark on the right side of FIG. 9B)during the display change at the timing of pressing the print bookmarkbutton does not become a print candidate.

FIG. 9D is a print candidate image in the case of pressing the printbookmark button at the D9 time point of FIG. 7C. In this case, in thesame way as described above, the composite image of the slideshowdisplay at the timing of pressing the print bookmark does not become aprint candidate, and instead the composite image of the slideshowdisplay which does not change at the display timing of before and afterD8 and D10, as well as the individual images comprising the compositeimage, become the print candidate.

FIG. 10 shows an image in the case of displaying on the display unit 110of the TV receiving apparatus 100 a displayed image comprising printcandidate images collectively in the case in which the print bookmarkbutton has been pressed at the D4 and D9 time points of FIG. 7C.

In this way, at Step S1502, as shown in FIG. 12, the decided printcandidate image is made to be a selection option of an image to beoutput, and is displayed in a list so that it can be selected by a user.From here, the process advances to Step S1503, and the print candidateimage and the number to be printed is selected. After the print imagesare decided in this way, the red key 157R of the remote controlapparatus 150 assigned to the print start button is pressed, andprinting at Step S1504 is performed. In this case, if the details ofprinting are desired to be set, detailed items are selected by theyellow key 157Y of the remote control apparatus 150.

Second Embodiment

In the first embodiment, print candidate images were set to beindividual images “viewable” at the time of pressing the print bookmarkbutton, or individual images “viewable” at the time of pressing theprint bookmark button and slideshow displays before and after the timeof pressing the print bookmark button. However, in the secondembodiment, the case of applying the print candidate selection mode 2when displaying a randomly superimposing type slideshow display shown inFIG. 8D, will be described together with the flow of the process ofoperation in FIG. 16.

The operation of the second embodiment is an example of making allimages of other slideshow displays included in “viewable” individualimages at the time of pressing the print bookmark button, as printcandidates.

In this case, not only before the pressing of a print button, but alsoafter pressing of the print button and the slideshow display is advancedfurther, images of a slideshow display until all individual imagesimmediately after the print button is pressed and which are “viewable”become “hidden and not viewable, or disappeared”.

First, the slideshow display advances, and, at some time point, the userpresses the print bookmark button using the remote control apparatus150. After starting processing, when pressing of the print bookmarkbutton is detected at Step S1601, the setting content of the slideshowdisplay is confirmed at Step S1602. Subsequently, the process advancesto Step S1603, at which step the selection mode of the determination ofprint candidates is checked. If the selection mode of the determinationof the print candidate is print candidate selection mode 2, the processadvances to Step S1604, and the newest display state table of the printcandidate buffer is registered in the print list. However, if not aprint candidate selection mode 2, other processing is performed at StepS1611.

In the case of print candidate selection mode 2, after performingregistration processing at Step S1604, the process advances to StepS1605. Then, by the display state table, the “viewable” individualimages are checked, and registered in the display image list. Further,at Step S1606, it is checked whether individual images registered in thedisplay image list is in a “viewable” state in a slideshow display imagegenerated by all by all other display state tables in the printcandidate buffer. Any display state table with at least one image in a“viewable” state is registered in the print candidate buffer.

Next, the process advances to Step S1607, where it is checked if thepresent slideshow display is in the “display is in the middle of change”state. If “display is in the middle of change”, it is investigated atStep S1612 whether there has been storage to the print candidate bufferby the slideshow display control unit 502. Then, in the case where therehas been storage, the process advances to Step S1613, and the newestdisplay state table stored in the print buffer is added to the printcandidate list. The process then advances to Step S1614, “viewable”individual images are detected, and the individual images areadditionally registered in the display image list. Then, after this,whether there has been storage to the print candidate buffer is checkedat Step S1608, and, if there has been storage, the process advances toStep S1609. At Step S1609, it is determined whether there is displayedin the newest display state table stored in the print candidate buffer,“viewable” individual images in the display image list. And, if there isany “viewable” individual image, the process advances to Step S1615, andadded to the print candidate buffer. At Step S1607, in the case of notbeing in a “display is in the middle of change” state, the sameoperation as described above is performed.

Then, the process returns to Step S1608, and re-storage to the printcandidate buffer is awaited. When individual images are no longerdetected in the display state table, it is checked at Step S1610 whetherslideshow display is finished or not. If not finished, the processreturns to Step S1601 of detecting the print bookmark button.

In addition, if the print bookmark button is again pressed in the middleof the operation routine of the randomly superimposing type slideshowdisplay, the process is canceled, and operation from Step. S1601 isagain performed. However, in this case, the content of the printcandidate list and the display image list remains as is.

In addition, if finished in the middle of operation of the randomlysuperimposing type slideshow display, the display is finished at thattime point, storage to the print image buffer is also finished, andgeneration of a print candidate list is also finished at this point. Inthis way, when slideshow display viewing is finished, operation isperformed according to FIG. 15, as in the first embodiment. That is, asshown in FIG. 15, after starting processing, the user first selects theprint button from the print menu displayed at Step S1501, and selectsthe print candidate button. Then, the process advances to Step S1502,where the state image of the slideshow display based on the “viewable”individual images from the display image list, and the display statetable from the print candidate list is reproduced, and displayed as aprint candidate image.

FIG. 12 shows the display candidate image displayed on the display unit110 of the TV receiving apparatus 100, when the print bookmark buttonhas been pressed at both the D′1 and D′8 time points of FIG. 8D. Thisprint candidate image is an image included in either FIG. 11A or FIG.11D. After listing the print candidate images, the images desired to beprinted and the number of prints is selected at Step S1503, and printingis started at Step S1504. In addition, in FIGS. 11A-11D, images with anX mark are not registered as print candidate images.

Other Embodiments

In the first and second embodiments above, print bookmarking isperformed during the display of a slideshow, and, after finishingslideshow display, the images that are print candidates are displayed atonce, and, of those images, the print image is selected. However, duringthe setting of the slideshow display, the print candidate selection modeis set to “direct printing”. Then, by the print candidate determinationprocedure of the print mode selection mode 1, each time the printbookmark button is pressed, the display operation of the slideshowdisplay is paused, and a mode to display the print candidate screen isprepared. A flow showing the operation in this case is shown in FIG. 17.

Description of parts that overlap with parts of the former embodiments 1and 2 will be omitted. That is, the operation from the pressing of theprint bookmark button to Steps S1701 through S1707 are the same as thefirst embodiment. However, after this, a pause is instructed against theslideshow display control unit 502 at Step S1708, and, after that, theprint start routine is started at Step S1709. The print start routineS1709, in here, displays as a print candidate image registered in theprint list, and performs printing operations. In addition, at Step S1713is processing of other modes, as in the above embodiments.

After this, the slideshow display control unit 502 checks whether theprint routine is finished at Step S1710. Here, if it is detected to befinished, the process advances to Step S1711, outputs an instruction torestart slideshow display, and continues slideshow display until theslideshow is finished at Step S1712. By doing so, it is possible toprint while viewing a slideshow.

The above embodiments of the present invention were described in termsof displaying a strip-shaped type slideshow and a randomly superimposingtype slideshow display, but it is possible to apply the presentinvention to slideshow displays of different formats. Moreover,individual images which make up a composite image are formed not only byimages from various image information sources, but it is also possibleto use the individual images mixed with images from various imageinformation sources such as broadcast signals, video information fromDVD, and video information accumulated in an HDD apparatus.

In addition, the object of the present invention can also be achieved bysupplying, to a system or an apparatus, a storage medium on which isrecorded program code of software implementing the functions of theabove embodiments. That is, it is needless to say that the above objectcan be achieved by reading out and executing program code stored in astorage medium by a computer (or CPU or MPU) of that system orapparatus. In this case, program code itself which has been read outfrom the storage medium realizes the function of the above embodiments,and the storage medium storing that program code makes up the presentinvention.

As a storage medium for supplying program code, it is possible to use,for example, a flexible disk, hard disk, optical disc, magnetic opticaldisk, CD-ROM, CD-R, electromagnetic tape, non-volatile semiconductormemory card, or ROM. In addition, there is a case in which the functionsof the above embodiments are realized by executing program code read outby a computer.

However, further, it is needless to say that the above embodimentsinclude the case where, based on instructions of program code, an OS(operating system), etc., operating on a computer performs part or allof the actual processing.

Further, it is also possible that program code that has been read outfrom a storage medium can be written into memory equipped to a functionexpansion board inserted in a computer or a function expansion unitconnected to a computer. Needless to say, the embodiments include thecase of a CPU, etc., equipped to that function expansion board orfunction expansion unit performing a part or all of the actualprocessing according to the program code instructions, and therebyrealizing the functions of the above embodiments.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2006-221245, filed on Aug. 14, 2006, and Japanese Patent Application No.2007-192651, filed Jul. 24, 2007, which are hereby incorporated byreference herein in their entirety.

1. An image processing apparatus, comprising: a display control unitconfigured to display on the display apparatus a composite imagecomprising a plurality of images while dynamically changing theplurality of images; an receiving unit configured to receive outputcommands during displaying of the composite image on the displayapparatus by the display control unit; and a selection unit configuredto display, so that a user can select as an image to be output, eitherthe composite image at a timing before or after a timing at which theoutput command has been received by the receiving unit.
 2. The imageprocessing apparatus according to claim 1, wherein the selection unit,after displaying on the display apparatus both composite images at atiming before and after a timing at which the output command wasreceived by the receiving unit, as well as a composite image at a latertiming, displays either the composite image at a timing before or aftera timing at which the output command has been received by the receivingunit.
 3. The image processing apparatus according to claim 1, whereinthe selection unit, as a selection option of an image to be output thatcan be selected, displays, by the dynamic change, a composite imageincluding an image that has been additionally displayed and a compositeimage not including an image that has been additionally displayed andwhich is a composite image before the dynamic change.
 4. The imageprocessing apparatus according to claim 1, wherein the selection unitfurther displays each of the plurality of images comprising thecomposite image as a selection option that can be selected as an imageto be output.
 5. The image processing apparatus according to claim 1,wherein a display control by the display control unit has a pause statebefore and after a dynamic change of a display, and the composite imageis a composite image which is displayed in the pause state.
 6. The imageprocessing apparatus according to claim 1, further comprises an outputunit configured to output a composite image selected through theselection unit to a printing apparatus.
 7. A control method of an imageprocessing apparatus, comprising: controlling step of controlling acomposite image comprising a plurality of images by displaying on adisplay apparatus while dynamically changing the plurality of images;receiving step of receiving an output command from a user whiledisplaying the composite image on the display apparatus by thecontrolling step; and selecting step of displaying the composite imagesat a timing before and after the timing at which the output command hasbeen received by the receiving step so that a user can select either oneof the composite images as an image to be output.
 8. A storage mediumcapable of being read by a computer and on which is recorded a programfor making a computer function as each unit in the image processingapparatus according to one of claim 1.